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ABSTRACT 

This report describes the> process followed by the 
National Center on Educational Outcomes (NCEO) in developing a 
comprehensive model of educational outcomes and indicators for all 
students, including students with disabilities, who are completing 
school. Stage One involved identifying five alternative models of 
educational outcomes — an educational model, a sociological model, a 
political/pragmatic model, a psychological model, and a developmental 
model — and proposing to develop outcomes and indicators as a function 
of disability severity and age level of the student. In Stage Two, 
the terms "outcomes" and "indicators" were defined, a set of 
assumptions to identify "outcomes indicators" was generated, and the 
work was reviewed and revised. Stage Three brought about more formal 
and broad-based review of the proposed model. Stage Four's focus was 
presentation of a new model which distinguished between "enabling 
outcomes" and "educational outcomes." In Stage Five, additional 
review and consensus^bui Iding took place, resulting in agreement on 
and publication of a final list of indicators of school completion 
outcomes. NCEO*s plans for the future are outlined. (JDD) 



^'^ Reproductions supplied by EDRS are the best that can be made * 
* from the original document. * 
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Developing a Model 
Educational Outcomes 

Describing the process and stages leading to the development of a comprehensive 
modei of educational outcomes and indicators for students completing school. 



by James Ysseldyke and Martha Thurlow 



The need for education to be 
acc-.- mtable for its effects on stu- 
dents v/ith disabilities was a cen- 
tral factor in forming the National 
Center on Educational Outcomes 
(NCEO). 

NCEO works with state and 
federal education agencies to 
create and promote the use of edu- 
cational outcomes and indicators 
for students with disabilities. 
Educators believe that the use of 
outcomes and indicators will 
improve educational results for 
all students. 
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NCEO has developed a compre- 
hensive conceptual model of edu- 
cational outcomes and produced 
documents that identify these out- 
comes and their indicators. For 
each of six developmental levels 
(ages 3 and 6, grades 4, 8, and 12, 
and post-school), NCEO has been 



publishing the model with a set of 
outcomes and indicators. 

This comprehensive model and 
these documents evolved slowly. 
It took NCEO months of working 
with state departments of educa- 
tion Dersonnel, federal agencies, 
professional associations, parents, 
and advocacy groups to achieve 
consensus on the kinds of data 
that ought to be collected on 
students. 

The following stages describe the 
process that NCEO experienced 
while developing its comprehen- 
sive model of educational out- 
comes and indicators. 

Stage One 

In the 1990 funding proposal sub- 
mitted to the Office of Special 
Education Programs for NCEO, 



five University of Minnesota 
education experts identified alter- 
native models of educational 
outcomes (Robert Bruininks, 
Stanley Deno, Kevin McGrew, 
Martha Thurlow, and James 
Ysseldyke). They proposed an 
educational model, a sociological 
model, a poUtical/pragmatic 
model, a psychological model, and 
a developmental model. For each 
model, they proposed to develop 
outcomes and indicators as a 
function of severity of disabiUty, 
and as a function of the age level 
of the student. 
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Stage Two 

After funding was awarded to the 
University in October, 1990, NCEO 
staff started the process of devel- 
oping the basic model. 

To begin, the staff defined the 
terms "outcomes" and "indica- 
tors" by reviewing numerous 
definitions proposed in the profes- 
sional literature by researchers 
and policymakers. They agreed 
that "outcomes" were the results 
of interactions among individuals 
and schooling experiences. To 



complete the definition, they 
defined what was meant by "inter- 
actions," "schooling experiences," 
"individuals," and "results." 

An "indicator" was defined as a 
symbolic representation of one or 
more educational outcomes for 
infants, children, and youth that 
enables comparisons to be made. 

Next, NCEO staff expanded on the 
meaning of the key terms "sym- 
bohc representations," "out- 
comes," and "comparisons," and 
generated a set of assumptions to 



help identify "outcomes indica- 
tors." These were: 

1, Indicators of outcomes in spe- 
cial education should be related, 
conceptually and statistically, to 
those that are useful in education. 

2, Educational ( -utcomes indica- 
tors must provide necessary data 
to make policy decisions primarily 
at school, state, and national 
levels, but possess implications for 
evaluating educational programs 
at other levels. 

3, Outcomes indicators are most 
useful when they are functionally 
related to educational inputs, 



Figure 1, Models 1 &2 
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contextual characteristics^ and 
processes. 

In February, 1991, NCEO staff met 
with six state directors of special 
education, NCEO gave the direc- 
tors a preliminary' draft of Working 
Paper 1 to review, which included 
definitions, assumptions, and new 
preliminary models (Figure 1)« 
Attending that meeting were: 
Patrick Campbell (C A), Wayne 
Erickson (MN), Mary-Beth Fafard 
(MA), Tom Gillung (CT), Linda 
Hargan (KY), and Dean Meyers 
(SD). These directors offered sev- 
eral suggestions to clarify defini- 
tions and reorganize the paper. 

Following that meeting, NCEO 
staff presented the revised 
definitions, assumptions, and pro- 



posed model at several confer- 
ences. One of these was the annual 
Leadership Conference for state 
directors of special education. 
Another was a conference spon- 
sored by the Connecticut 
Department of Education where 
participants examined the future 
of education for students with dis- 
abilities. 

At the Connecticut meeting, 
NCEO staff received feedback on 
the model and indicators from 
Jerry Spears, Jim Wade, Frank 
Rusch, personnel in the 
Connecticut Department of 
Education, and others attending 
the conference. The resulting 
paper, Expected Educational 
Outcomes for Students with 
Disabilities, was published in 1992 



by James Ysseldyke, Martha 
Thurlow, and Robert Bruininks, 

Stage Three 

NCEO personnel revised Working 
Paper V and sent it to selected rep- 
resentatives of professional associ- 
ations and to all state directors of 
special education. It was formally 
reviewed at meetings in Montana, 
Michigan, and Nebraska, 
Responses to the paper and its 
models were gathered during a 
six-month period and synthesized 
in a document entitled Synthesis of 
Responses to Working Paper 1* 

The resulting model (Figure 2) was 
reviewed by the NCEO National 
Advisory Conunittee. This model 



Figure 2, Model 3 
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Development of a Comprehensive Model of Educational Outcomes and Indicators 



introduced the notion of "enabling 
outcomes" in response to the need 
to recognize intermediate out- 
comes such as learning Braille or 
reading sign language. 

In December, 1991, NCEO con- 
vened a meeting of individuals 
who had participated in a number 
of activities that were relevant to 
the development of outcomes and 
indicators. Attending that meeting 
were Bob Algozzine (University of 
North Carolina at Charlotte), Peter 
Behuniak (Connecticut 
Department of Education), Louis 
Danielson (U.S. Office of Special 
Education Programs), Ruth Flynn 
(Missouri Department of 
Education), William Frey 
(Disability Research Systems in 
Michigan), Harvey Harkness 
(New Hampshire Department of 
Education), Robin Kimbrough - 
(American Public Welfare 
Association), Kenneth Olsen (Mid- 
South RRC), Patricia Sitlington 
(Iowa Department of Education), 
and Jennifer York (University of 
Minnesota). 

At this December meeting, NCEO 
used a multi-attribute consensus- 
building process to help those 
attending generate, edit, and reach 
agreement on outcomes and indi- 
cators in each of the domains in 
the outcomes model. 

The following April, 1992, NCEO 
staff members met with a group of 
Minnesota parents and school 
personnel. They conducted a 
multi-attribute consensus-building 
process with the group in an effort 
to have those individuals generate 
outcomes and indicators, and 
point out their importance. In 
attendance at that meeting were 



Jane Ciromie, Rick Green, Marie 
Knowlton, Nancy Larson, 
Kathleen Steffens, and Kyla 
Wahlstrom. 

Also that April, NCEO personnel 
presented the revised model at the 
annual Leadership Conference 
held by the Office of Special 
Education Programs. In the pre- 
sentation, NCEO included the def- 
initions of the terms "outcomes" 
and "indicators," stated assump- . 
tions, and described unresolved 
issues. NCEO received reactions 
from quite a few participants, 
including state directors of special 
education. 

In May 1992, NCEO met in 
Minneapolis with the following 
five state directors of special edu- 
cation: Stevan Kukic (UT), Wayne 
Erickson (MN), Nancy Thabet 
(WV), Bob Kennedy (NH), and 
Mary-Beth Fafard (MA). NCEO 
again used a multi-attribute con- 
sensus-building process to help 
participants reach agreement on 
indicators for one of the domains 
in the model (literacy) at grade 12. 
The meeting generated plans for 
developing indicators ir the 
future, with a recommendation to 
eliminate repeating the consensus- 
building process with many 
groups. 

Instead, participants recommend- 
ed that a group with comprehen- 
sive and national representation 
be brought together for a consen- 
sus-building session to produce a 
final list of outcomes and indica- 
tors at one developmental level. 
NCEO selected the school comple- 
tion level (grade 12) for the first 
outcomes and indicators 
document. 



Stage Four 

In August, 1992, NCEO published 
Working Paper 2^ In that paper, a 
new model of educadonal out- 
comes appeared (Figure 3). This 
model redefined the "enabling 
outcomes" and "educational 
outcomes/' 

"Educational outcomes"''were 
defined as the result of interac- 
tions between individuals and 
schooling experiences, "Enabling 
outcomes" were the result of inter- 
actions between individuals and 
life experiences that provide the 
individual with the opportunity to 
reach educational outcomes. 

In Working Paper 2, the four 
enabling outcomes originally iden- 
tified in the model (presence and 
participation, compensatory and 
accommodation skills, adaptive 
behavior, family coping and sup- 
port skills) were reduced to two 
(presence/participation and 
accommodation/ adaptation/ com- 
pensation). In addition, the names 
of the six educational outcomes 
shifted Slightly 

Stage Five 

In September, 1992, NCEO staff 
members published a paper based 
on the model in Teaching 
Exceptional Children, They also con- 
vened a meeting in Washington, 
DC, that was attended by repre- 
sentatives of more than 20 agen- 
cies or groups. The meeting once 
again used the multi-attribute con- 
sensus-building process. Its pur- 
pose was to reach agreement on a 
final list of indicators of school 
completion outcomes. 
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Figure 3, Model 4 
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Development of a Comprehensive Model of Educational Outcomes and Indicators 



Those in attendance reviewed the 
outcomes and indicators and as a 
result, the model shifted again 
(Figure 4), The new changes were 
compiled into a document draft 
entitled Educational Outcomes and 
Indicators for Students Completing 
School* After several selected 
directors of special education 
reviewed the document, modifica- 
tions were made and it was pub- 
lished in January, 1993. 

Moving Ahead 

NCEO staff are identifying sources 
of data for indicators, distributing 
a self-study guide for use by states 



and districts, and working with 
state education agencies to devel- 
op alternative uses of the model. 

During the summer of 1993, one 
state used the model to aggregate 
progress data on Individualized 
Education Program (lEP) objec- 
tives. Two states proposed aggre- 
gating progress data on lEP objec- 
tives. Two other states proposed 
aggregating data on the perfor- 
mance of students with disabili- 
ties, and another state used the 
model to develop transition lEPs, 

NCEO developed outcomes and 
indicators for the post-school 



level, and for ages three and six, 
changing the model slightly at 
each level In 1994, NCEO will 
develop outcomes and indicators 
for grades four and eight. ♦ 



* Publication available for purchase by writing 
NCEO Publications Office. 
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Figure 4 

Conceptual Model of Outcomes 
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